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Previous  research  on  strategy  selection  in  dynamic  task  environments  indicated  that 
subjects  preferred  to  request  information  first,  before  an  action  was  applied,  even 
when  the  straightforward  application  of  actions  would  have  resulted  in  more  optimal 
performance.  Furthermore,  this  strategy  was  also  used  when  subjects  only  had  limited 
time  for  diagnosis.  In  the  present  experiment  it  was  investigated  whether  the  amount 
of  training  could  account  for  the  limited  use  that  subjects  made  of  the  task  dynamics. 
Subjects  were  required  to  monitor  the  changing  fitness  level  of  an  athlete,  by  means 
of  a  graph  on  a  computer  screen,  and  to  apply  treatments  whenever  necessary.  They 
could  request  various  symptoms  that  would  provide  an  indication  for  the  cause 
underlying  a  possible  fitness  decline.  The  subjects  either  received  limited  training  or 
elaborate  training  and  they  had  either  sufficient  time  for  diagnosis,  or  worked  under 
time  pressure. 

The  results  showed  that  the  amount  of  training  did  not  affect  the  strategy  that 
subjects  used:  half  of  the  subjects  used  a  judgment-oriented  strategy  and  the  other 
half  used  an  action-oriented  strategy  in  both  training  conditions.  However,  the  well 
trained  subjects  were  superior  in  selecting  information  and  they  processed  the 
information  at  a  faster  rate  than  the  subjects  with  only  minimal  training.  The  time 
pressure  effects  replicated  previous  findings:  subjects  used  the  same  strategy  and 
speeded  up  information  processing.  A  high  level  of  time  pressure  only  deteriorated 
the  information  integration  processes  of  the  subjects  who  had  received  only  minimal 
training,  but  not  that  of  the  well-trained  subjects. 

The  findings  suggest  that  training  alone,  without  any  instructions  on  how  to  deal  with 
the  task,  does  not  result  in  the  selection  of  an  optimal  decision  strategy,  posing 
linutations  to  the  adaptivity  of  decision  making  behaviour  in  a  dynamic  task 
environment.  Furthermore,  the  results  suggest  that  elaborate  training  can  overcome 
negative  time  pressure  effects,  especially  with  respect  to  the  integration  of  informati¬ 
on. 


*  Per  I  februari  1994  is  de  naam  Instituut  voor  Zintuigfysiologie  TNO  gewijzigd  in  TNO  Technische 
Menskunde. 
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SUMMARY 

Previous  research  on  strategy  selection  in  dynamic  task  environments  indicated 
that  subjects  preferred  to  request  information  first,  before  an  action  was  applied, 
even  when  the  straightforward  application  of  actions  would  have  resulted  in 
more  optimal  performance.  Furthermore,  this  strategy  was  also  used  when 
subjects  only  had  limited  time  for  diagnosis.  In  the  present  experiment  it  was 
investigated  whether  the  amount  of  training  could  account  for  the  limited  use 
that  subjects  made  of  the  task  dynamics.  Subjects  were  required  to  monitor  the 
changing  fitness  level  of  an  athlete,  by  means  of  a  graph  on  a  computer  screen, 
and  to  apply  treatments  whenever  necessary.  They  could  request  various  symp¬ 
toms  that  would  provide  an  indication  for  the  cause  underlying  a  possible  fitness 
decline.  The  subjects  either  received  limited  training  or  elaborate  training  and 
they  had  either  sufficient  time  for  diagnosis,  or  worked  under  time  pressure. 

The  results  showed  that  the  amount  of  training  did  not  affect  the  strategy  that 
subjects  used:  half  of  the  subjects  used  a  judgment-oriented  strategy  and  the 
other  half  used  an  action-oriented  strategy  in  both  training  conditions.  However, 
the  well  trained  subjects  were  superior  in  selecting  information  and  they  pro¬ 
cessed  the  information  at  a  faster  rate  than  the  subjects  with  only  minimal 
training.  The  time  pressure  effects  replicated  previous  findings:  subjects  used  the 
same  strategy  and  speeded  up  information  processing.  A  high  level  of  time 
pressure  only  deteriorated  the  information  integration  processes  of  the  subjects 
who  had  received  only  minimal  training,  but  not  that  of  the  well-trained  subjects. 
The  findings  suggest  that  training  alone,  without  any  instructions  on  how  to  deal 
with  the  task,  does  not  result  in  the  selection  of  an  optimal  decision  strategy, 
posing  limitations  to  the  adaptivity  of  decision  making  behaviour  in  a  dynamic 
task  environment.  Furthermore,  the  results  suggest  that  elaborate  training  can 
overcome  negative  time  pressure  effects,  especially  with  respect  to  the  integra¬ 
tion  of  information. 
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Rap.nr.  TNO-TM  1994  B-12 


TNO  Technische  Menskunde‘ 
Soesterberg 


Het  effect  van  training  op  de  selectie  van  een  beslisstrategie  in  een  dynamische 
taakomgeving  als  functie  van  tydsdruk 

J.H.  Kerstholt 
SAMENVATTING 

Voorgaand  onderzoek  naar  de  beslisstrategieen  die  in  dynamische  taken  worden 
gebruikt,  heeft  aangetoond  dat  proefpersonen  bij  voorkeur  eerst  informatie 
opvragen  voordat  zij  een  actie  uitvoeren,  zelfs  in  taken  waar  het  louter  uitvoeren 
van  acties  tot  een  betere  uitkomst  zou  hebben  geleid.  Een  dergelijke 
diagnostische  strategic  werd  ook  gebruikt  als  de  proefpersonen  slechts  weinig  tijd 
hadden  voor  het  nemen  van  een  bcslissing.  In  het  huidige  experiment  werd 
onderzocht  in  hoeverre  de  hoeveelheid  training  een  invloed  heeft  op  de  mate 
waarin  men  gebruik  maakt  van  de  continue  feedback  over  de  toestand  van  het 
systeem.  Proefpersonen  moesten  het  veranderende  conditieniveau  van  een  atleet 
in  de  gaten  houden  en,  inJien  dit  nodig  was,  moesten  zij  ook  behandelingen 
geven  aan  de  atleet.  De  proefpersonen  konden  verschillende  symptomen 
opvragen,  hetgeen  een  indicatie  verschafte  voor  de  oorzaak  van  de 
conditieverslechtering.  De  helft  van  de  proefpersonen  kreeg  slechts  een  beperkte 
training  en  de  andere  helft  van  de  proefpersonen  kreeg  een  uitgebreidere 
training.  Zij  werkten  ofwel  onder  tijdsdruk,  of  zij  hadden  voldoende  tijd  voor  een 
diagnose. 

De  resultaten  tonen  aan  dat  de  hoeveelheid  training  geen  effect  had  op  de 
gebruikte  strategie;  in  beide  trainingscondities  richtte  de  helft  van  de 
proefpersonen  zich  op  de  diagnose  van  de  conditieverslechtering,  en  de  andere 
helft  paste  alleen  behandelingen  toe.  De  proefpersonen  die  goed  getraind  waren 
selekteerden  echter  meer  diagnostische  informatie  en  zij  verwerkten  de 
informatie  sneller  dan  de  groep  die  een  beperkte  training  had  gehad.  Tijdsdruk 
verslechterde  de  integratie  van  informatie  alleen  voor  de  proefpersonen  die 
slechts  minimaal  waren  getraind. 

De  resultaten  laten  zien  dat  training  alleen,  dus  zonder  aanwijzingen  over  de 
manier  waarop  de  taak  aangepakt  zou  moeten  worden,  niet  re.sulteert  in  het 
gebruik  van  een  optimale  strategie.  Bovendien  wordt  aangetoond  dat  met  een 
uitgebreide  training  de  negatieve  effecten  van  tijdsdruk  voorkomen  kunnen 
worden,  met  name  voor  wat  betreft  de  integratie  van  informatie. 


'Per  1  februari  1994  is  de  naam  Instituut  voor  Zintuigfysiologie  TNO  gewijzigd  in  TNO  Technische 
Menskunde. 
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I  INTRODUCTION 

Even  though  many  real-world  decision  tasks  are  of  a  dynamic  nature,  such  as 
plant  control,  car  driving  or  patient  care,  research  has  paid  only  little  attention 
to  the  cognitive  processes  underlying  dynamic  decision  making  (Brehmer,  1992; 
Hammond,  1988). 

One  of  the  main  characteristics  of  dynamic  tasks,  distinguishing  them  from  the 
commonly  used  static  tasks,  is  that  the  process  to  be  controlled  changes  over 
time.  This  task  feature  implies  that  a  decision  maker  not  only  has  to  decide  how 
to  react  to  system  changes,  but  also  when  to  react.  Thus,  when  the  operator  in  a 
power  plant  observes  from  his  control  panel  a  decline  in  system  performance,  he 
can  react  immediately,  but  he  can  also  wait  and  observe  the  developments  for  a 
while,  in  order  to  determine  the  need  for  active  interference. 

Furthermore,  dynamic  environments  often  provide  some  global  indication  of 
system  performance  which  can  be  used  to  evaluate  the  effects  of  executed 
actions.  In  other  words,  the  decision  maker  receives  feedback,  which  allows  him 
or  her  to  solve  the  problem  incrementally,  i.e.  taking  only  small  steps  at  a  time 
(Hogarth,  1981).  This  possibility  to  receive  feedback  elaborates  the  s'*t  of 
decision  strategies  in  dynamic  situations  with  the  so-called  ’action-oriented’ 
strategies  (Kleinmuntz  &  Thomas,  1987):  an  action  can  be  applied,  its  effect  can 
be  observed  from  the  overall  system  performance,  and  if  the  desired 
improvement  is  not  shown,  the  decision  maker  can  continue  with  other  actions 
until  his  or  her  goal  is  attained.  Previous  studies  showed,  however,  that  subjects 
preferred  a  ’judgment-oriented’  strategy,  even  though  an  action-oriented  strategy 
would  have  resulted  in  better  performance:  preceding  the  implementation  of  an 
action  subjects  first  requested  information  in  order  to  learn  about  the  cause 
underlying  the  decline  in  system  performance  (Kerstholt,  submitted;  Kleinmuntz, 
1985;  Kleinmuntz  &  Thomas,  1987).  Thus,  in  a  dynamic  environment  subjects 
seem  not  to  select  the  strategy  that  provides  both  the  best  outcome  and  which 
avoids  the  effortful  information  search  and  integration  processes.  This  result 
contrasts  with  the  overall  conclusion  of  Payne,  Bettman  and  Johnson  (1993),  who 
state  that,  to  a  large  extent,  decision  makers  make  use  of  the  environmental 
structure  and  select  a  strategy  that  performs  fairly  well  in  terms  of  the 
combination  of  effort  and  accuracy.  One  possible  explanation  for  these  opposing 
results  is  that  adaptivity  only  concerns  static  tasks  and  well  structured  decision 
problems,  and  not  dynamic  tasks,  which  are  in  general  somewhat  more  complex 
to  deal  with.  Another  possibility,  which  will  be  investigated  in  the  present 
experiment,  refers  to  the  experience  subjects  have  with  the  task.  A  prerequisite 
for  adaptive  responses  is  that  people  know  the  relation  between  strategy  and 
task  performance:  the  most  efBcient  strategy  can  only  be  adopted  when  the 
effect  of  behavioral  responses  on  outcome  performance  is  known.  In  previous 
studies  subjects  received  information  on  all  relevant  task  parameters,  but  they 
only  had  minimal  experience  with  the  actual  task.  In  the  present  study  we 
manipulated  the  amount  of  training  subjects  received,  and  designed  the  task  in 
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such  a  way  that  an  action-oriented  strategy  would  result  in  the  best  outcome 
performance. 

I  predicted  that  subjects  who  had  received  the  more  elaborate  training,  and 
would  consequently  be  able  to  learn  the  relation  between  the  use  of  a  strategy 
and  its  outcome,  would  make  use  of  the  task  dynamics,  i.e.  use  an  action- 
oriented  strategy. 

The  changing  nature  of  dynamic  tasks  implies  that  the  time  dimension  is  an 
important  factor  in  the  decision  process.  Not  only  do  decision  makers  need  to 
plan  the  time  span  of  the  diagnosis  process  and  the  time  at  which  an  action  is 
applied,  but  they  may  also  face  time  pressure  when  system  performance 
deteriorates  very  rapidly.  The  effects  of  time  pressure  on  decision  making 
behaviour  have  mostly  been  investigated  in  static  tasks,  in  which  decision 
problems  do  not  change  over  time  (see  tor  a  review  Svenson  &  Maule,  1993). 
Consistent  findings  from  this  line  of  research  are  that  decision  makers  speed  up 
information  processing  under  time  pressure,  they  request  less  information,  they 
use  a  different  kind  of  information,  and  they  resort  to  less  time  consuming, 
simpler  decision  strategies  (Edland  &  Svenson,  1993;  Payne,  Bettman  & 
Johnson,  1988). 

In  previous  experiments  in  which  we  used  a  dynamic  task  these  findings  were 
partly  replicated,  but  the  basic  strategy,  how  subjects  dealt  with  the  task,  did  not 
change  under  time  pressure,  i.e.  even  when  only  limited  time  was  a\'ailable 
subjects  used  a  judgment-oriented  strategy.  In  these  studies  we  used  a  within- 
subjects  design:  all  subjects  had  to  make  decisions  under  various  levels  of  time 
pressure.  A  possible  drawback  of  using  such  a  design  is  that  subjects  adopted  a 
strategy  that  would  result  in  reasonably  accurate  performance  over  all  time 
pressure  conditions.  Thus,  rather  than  planning  different,  optimal,  strategies  for 
each  time  pressure  condition  separately,  subjects  may  have  aimed  at  finding  one 
global  strategy  that  works  reasonably  well  over  all  time  pressure  conditions.  By 
using  a  between-subjects  design,  however,  subjects  would  be  able  to  select  a 
strategy  that  would  be  optimal  for  the  specific  time  span.  Thus,  on  the 
assumption  that  individual  subjects  face  only  one  type  of  time  limitation,  I 
predicted  a  strategy  switch  over  different  time  pressure  conditions. 

A  third  research  question  concerns  the  interaction  between  training  and  time 
pressure.  Many  professional  groups,  such  as  fireground  commanders  and  pilots, 
are  well  trained  in  procedures  that  should  be  used  in  crisis  situations  with  high 
levels  of  time  pressure  (Means,  Salas,  Crandall  &  Jacobs,  1993).  As  a  result  of 
such  an  extensive  training,  decision  makers  may  recognize  the  situation  as 
belonging  to  a  particular  class,  which  will  automatically  trigger  the  appropriate 
procedures  (Klein,  1993).  A  major  advantage  of  such  immediately  applicable 
knowledge  is  that  rapid  decisions  can  be  made,  and  unadaptive  reactions  due  to 
stress  can  be  avoided.  Thus,  unadaptive  reactions  to  time  pressure  may  be 
mediated  by  training:  when  a  decision  maker  appraises  his  or  her  capacity  or 
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knowledge  as  sufficient  to  deal  with  a  ocular  task,  negative  effects  of  time 
stress  may  be  overcome  (Maule  &  Hockey,  1993).  For  the  present  experiment  I 
predicted  that  performance  would  be  less  affected  by  high  levels  of  time  pressure 
when  subjects  were  well  trained  as  compared  with  subjects  who  did  not  have 
such  an  elaborate  training. 


2  METHOD 
2.1  Subjects 

Forty  subjects  voluntarily  participated  in  the  experiment.  They  were  students  at 
the  University  of  Utrecht,  and  were  paid  Dfl.  45  for  participation.  A  bonus  of  10 
guilders  was  given  to  the  5  best  performing  subjects. 


2.2  The  experimental  task 


Subjects  were  required  to  imagine  that  they  were  the  personal  attendant  of  an 
athlete  who  was  running  a  race.  The  fitness  level  of  this  athlete  was  presented  to 
the  subjects  by  means  of  a  graph  on  a  computer  screen  (see  Fig.  1).  This 
information  was  constantly  available  to  the  subjects.  The  fitness  level  of  the 
athlete  could  vary  between  100  (optimal  fitness  level)  and  0  (the  athlete  has 
collapsed). 


red 

yes 

sick 

? 

tired 

? 

thirsty 

no 

cool 

rest 

drink 

Fig.  1  Example  of  a  computer  screen  that  subjects  faced  in  the  experiment. 
The  graph  represents  the  athlete’s  fitness  level,  in  the  left  window  it  can  be 
seen  that  two  information  requests  have  been  made,  red  and  thirsty,  and  on 
the  bottom  the  three  possible  actions  are  shown. 
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At  several  points  in  time  the  fitness  level  would  start  to  decline,  which  could  be 
due  to  four  causes,  a  temperature  problem,  a  circulation  problem,  a  metabolism 
problem  or  a  false  alarm.  A  false  alarm  meant  that  the  fitness  level  of  the 
athlete  declined  without  any  physiological  cause,  i.e.  a  temperature,  a  circulation 
or  a  metabolism  problem,  and  it  would  therefore  recover  spontaneously  after 
some  time  (at  a  fitness  level  of  50).  When  the  decline  of  the  athlete’s  fitness 
level  was  caused  by  either  a  temperature,  a  circulation  or  a  metabolism  problem, 
on  the  other  hand,  the  athlete’s  fitness  level  would  decline  until  a  collapse 
(fitness  level  of  0),  assuming  that  nothing  was  done  by  the  subject.  The  various 
causes  of  a  decline  occurred  with  different  a  priori  probabilities:  a  temperature 
and  a  circulation  problem  occurred  with  a  probability  of  .1,  a  metabolism 
problem  with  a  probability  of  .3  and  a  false  alarm  with  a  probability  of  .5. 

A  decline  was  always  prompted  by  a  change  in  one  parameter,  multiple  causes 
were  excluded.  Furthermore,  the  decline  evolved  linearly  with  its  slope 
dependent  on  the  time  pressure  condition.  Thus,  the  graph  that  was  shown  on 
the  computer  screen  provided  information  on  the  onset  of  a  disturbance  (the 
fitness  level  starts  to  decline)  and  over  time,  subjects  would  learn  whether  the 
decline  was  merely  a  false  alarm  (the  athlete’s  fitness  level  spontaneously 
recovers)  or  was  caused  by  a  physiological  disturbance. 

Each  cause  resulted  in  a  specific  pattern  of  symptoms.  The  subjects  could 
request  this  information  in  order  to  determine  the  specific  cause  underlying  the 
fitness  decline.  The  information  requests  were  served  by  mouse  clicks  and  the 
response  was  either  ’yes’  (the  athlete  has  the  symptom)  or  ’no’  (the  athlete  does 
not  have  the  symptom).  The  symptoms  that  could  be  requested  were:  red,  feeling 
sick,  tired,  and  thirsty.  The  probabilities  of  the  occurrence  of  a  symptom,  given  a 
particular  cause  were  as  follows  (the  probability  of  the  symptom,  given  other 
causes  [p(Si)  |  -Hj]  is  put  in  brackets): 


temperature 

circulation 

metabolism 

no  problem 

red  colour 

0.9  (0.2) 

0.1  (0.3) 

0.2  (0.3) 

0.2  (0.3) 

feeling  sick 

0.2  (0.3) 

0.8  (0.2) 

0.5  (0.2) 

01  (0.5) 

tired 

0.3  (0.5) 

0.4  (0.5) 

0.6  (0.5) 

0.5  (0.5) 

thirsty 

0.3  (0.4) 

0.2  (0.4) 

0.7  (0.3) 

0.3  (0.5) 

If  a  decline  was  caused  by  a  physiological  disturbance  the  subjects  needed  to 
apply  a  treatment  in  order  to  prevent  the  athlete  from  collapsing.  For  each 
problem  one  specific  action  was  needed: 
in  case  of  a  temperature  problem:  to  cool 

in  case  of  a  circulation  problem:  to  rest 

in  case  of  a  metabolism  problem;  to  drink 
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If  the  correct  treatment  was  applied  the  fitness  level  would  be  restored,  which 
could  be  deduced  from  a  change  in  the  curve  from  a  decreasing  fitness  level  to 
an  increasing  or^- 


2.3  Normative  strategy:  Monte  Carlo  simulations 

A  computer  program  was  written  that  simulated  two  decision  strategies:  a 
judgment-oriented  strategy  and  an  action-oriented  strategy. 

Judgment-oriented  strategy’ 

For  the  judgment-oriented  strategy  the  program  started  with  considering  all 
symptom  information  when  the  athlete’s  fitness  level  declined.  Based  on  this 
information  the  a  posteriori  probability  of  each  hypothesis  was  calculated,  of  a 
temperature  problem,  of  a  circulation  problem,  of  a  metabolism  problem  and  of 
a  false  alarm.  The  program  then  selected  the  hypothesis  with  the  highest 
probability.  The  selection  of  one  hypothesis  implied  the  application  of  one  single 
treatment,  which  was  then  applied.  If  the  treatment  was  correct  the  trial  ended, 
and  if  the  treatment  was  incorrect,  i.e.  the  fitness  level  would  continue  to 
decline,  the  next,  second  best,  hypothesis  was  selected,  the  appropriate  treatment 
applied  and  its  effect  determined.  This  process  continued  until  the  athlete’s 
fitness  level  spontaneously  recovered  (in  the  case  of  a  false  alarm),  or  when  the 
correct  treatment  had  been  applied  or  when  the  athlete  had  collapsed. 

Action-oriented  strategy 

For  this  strategy  the  program  only  .selected  treatments  as  a  reaction  to  a  fitness 
decline  (in  a  fixed  order:  drink,  cool,  and  rest).  After  applying  a  treatment  its 
effect  was  determined,  and  if  the  fitness  level  was  not  increasing,  the  following 
treatment  was  applied.  A  trial  would  end  when  the  fitness  level  spontaneously 
increased  (in  case  of  a  false  alarm),  when  the  correct  treatment  had  been 
applied  or  when  the  athlete  collapsed. 

These  strategies  were  run  in  the  task  environment  that  has  been  described 
above. 

The  performance  of  both  decision  strategies  was  expressed  by  the  overall  money 
supply  at  the  end  of  a  run.  Each  time  information  was  requested  0,50  guilder 
was  deducted  from  the  overall  money  supply,  for  the  application  of  an  action  1,0 
guilders  and  for  an  athlete  collapse  5  guilders.  For  a  correct  treatment  2,50 
guilders  were  added  to  the  money  supply.  Furthermore,  it  was  simulated  that  an 
information  request  would  take  1  second  and  that  the  selection  of  an  action 
would  take  5  seconds. 
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The  simulations  of  both  decision  strategies  were  run  5000  times  in  6 
experimental  conditions:  the  available  time  could  either  be  25,  12.5  or  8.3 
seconds,  and  decision  activities  could  either  begin  at  a  fitness  level  of  75  or  at  a 
fitness  level  of  50. 

The  results  of  these  simulations  are  shown  in  Fig.  2  (mean  amount  of  money  per 
trial,  corresponding  to  one  fitness  decline). 


low  time  moderate  time  high  time 
pressure  pressure  pressure 


Fig.  2  Outcome  (mean  amount  of  money  per  trial)  of  the  Monte 
Carlo  simulations,  for  the  judgemen;-oriented  and  the  action-oriented 
strategy,  which  could  be  applied  at  a  fitness  level  of  either  50  or  75, 
in  three  time  pressure  conditions  (low,  moderate  and  high). 


In  each  time  pressure  condition  the  action-oriented  strategy  resulted  in  better 
performance;  the  expected  benefit  per  .  m  is  higher.  Furthermore,  it  appears  to 
be  more  advantageous  to  wait  until  a  fitness  level  of  50,  rather  than  to  start  with 
interference  at  a  fitness  level  of  75.  Thus,  it  can  be  concluded  from  these 

simulations  that  in  this  specific  task,  it  would  be  optimal  to  start  interference  at 

a  fitness  level  of  50,  and  to  apply  actions  only.  The  results  of  the  simulation  are 
to  some  extent  dependent  on  the  selection  of  the  times  for  an  information 

request  and  the  application  of  a  treatment.  Note,  however,  that  the  simulations 

scheduled  5  seconds  for  the  application  of  an  action  and  only  1  second  for  an 
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information  request,  which  implies  that  the  chosen  times  formed  already  a 
disadvantage  to  the  action-oriented  strategies. 


2.4  Procedure 

Tire  experiment  was  divided  into  three  parts:  two  training  sessions  and  the  actual 
experiment.  In  the  first  training  session  subjects  had  to  learn  the  relations 
between  combinations  of  symptoms  and  the  most  probable  causes.  This  training 
was  the  same  for  each  subject.  They  were  given  the  information  on  a  priori 
probabilities,  on  the  probabilities  of  symptoms  given  the  possible  causes  of  the 
decline.  [p(Si  ]  Hj)],  and  on  the  probabilities  of  the  symptoms  given  other 
possible  causes  of  the  decline,  (p(Si  |  -Hj)].  The  information  on  symptom/- 
hypothesis  relations  was  presented  in  eight  bar-plots.  Fig.  3  shows  an  example  of 
such  a  plot  for  one  of  the  symptoms  (red  colour).  Subjects  interacted  with  a 
computer  program  that  presented  them  with  random  combinations  of  symptoms 
(for  example,  "not  red,  not  sick,  thirsty  and  tired").  The  subject  had  to  select  the 
most  probable  cause  given  the  symptoms.  After  each  trial  they  were  given 
feedback  on  the  accuracy  of  their  diagnosis  and  in  case  of  an  incorrect  diagnosis 
they  were  also  told  which  one  should  have  been  selected.  After  each  run  of  10 
trials  the  subject  was  given  feedback  on  his  or  her  overall  score  of  the  run.  The 
general  learning  criterion  was  three  successive  runs  comprising  two  runs  that 
were  100%  correct. 


red  colour 


hypothesis  hypothesis 

Fig.  3  Information  provided  to  the  suWects  indicating  the  relations 
between  symptoms  and  possible  causes  or  a  fitness  decline. 

After  learning  the  relations  between  symptoms  and  underlying  causes,  the 
subjects  continued  with  the  training  of  the  actual  task.  This  training  session 
comprised  two  conditions,  a  minimal  training  condition  and  an  elaborate  training 
condition.  Half  of  the  subjects  practised  the  task  for  15  minutes  without 
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receiving  feedback  on  the  effects  of  their  actions  on  the  overall  money  level.  The 
other  half  of  the  subjects  went  through  an  elaborate  training  session:  they 
practised  for  30  minutes  and  were  continuously  given  information  on  the  overall 
money  supply.  This  way,  they  could  learn  the  effect  of  the  used  strategy  on  their 
goal  to  end  with  the  highest  possible  amount  of  money.  The  subjects  started  off 
with  an  amount  of  Dfl.  100.-.  They  paid  Dfl.  0.50  for  one  information  request, 
Dfl.  1  for  the  application  of  a  treatment,  and  Dfl.  5  for  an  athlete  collapse.  They 
received  Dfl.  2.50  whenever  they  restored  the  athlete’s  fitness  level  correctly. 


2.5  Design 

A  2*2  (time  pressure,  training)  factorial  was  used.  Time  pressure  was 
manipulated  by  the  slope  of  system  decline.  It  was  a  between  subjects  factor, 
each  subject  was  in  either  the  slowly  changing  condition  or  in  the  quickly 
changing  condition.  There  were  two  time  pressure  conditions;  low  time  pressure 
(slope  of  4,  implying  that  the  athlete’s  fitness  level  would  decrease  in  25  seconds 
from  100  to  0)  and  high  time  pressure  (slope  of  -12  or  8.3  seconds). 
Furthermore,  half  of  the  subjects  received  the  short  training  and  the  other  half 
of  the  subjects  received  the  elaborate  training. 

The  subjects  were  presented  with  30  fitness  declines  in  each  time  pressure 
condition.  Half  of  these  declines  were  caused  by  false  alarms.  After  a  correct 
treatment  was  applied  the  fitness  level  would  increase  to  a  value  of  100  and  the 
next  trial  would  start. 

Dependent  variables 

Timing,  is  indicated  by  the  time  span  between  the  onset  of  a  fitness  decline  and 
the  first  information  request  or  action.  Since  I  am  interested  in  a  strategic  effect, 
I  will  consider  the  relative  time,  i.e.  the  absolute  time  divided  by  the  available 
time. 

Decision  strategy,  is  indicated  by  the  ratio  between  the  number  of  times 
information  was  requested  after  the  onset  of  a  fitness  decline  and  the  number  of 
times  a  treatment  was  applied  immediately  after  the  onset  of  a  decline. 

Information  search,  is  indicated  by  the  number  of  information  requests  and  the 
kind  of  information  requests.  In  order  to  investigate  the  quality  of  the  requested 
information  we  calculated  the  uncertainty  reduction  with  respect  to  the 
underlying  causes  (Hj)  that  would  be  realised  when  each  of  the  symptoms  (S;) 
was  requested.  The  following  formula  was  used  (Dretske,  1981): 


i 
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This  formula  was  applied  for  both  the  situation  where  the  symptom  would  be 
absent  and  the  situation  where  the  symptom  would  be  present.  The  overall  value 
was  attained  by  weighing  the  amount  of  information  by  the  probability  of  the 
symptom  being  respectively  absent  or  present.  The  uncertainty  remaining  after 
knowing  the  outcome  for  one  symptom  (the  one  first  requested)  was  1.48  bits  for 
the  symptom  ’sick’,  1.53  bits  for  the  symptom  ’red’,  1.56  bits  for  the  symptom 
’thirst’  and  1.66  bits  for  the  symptom  ’tired’.  Thus,  from  these  data  it  can  be 
inferred  that  the  symptom  ’sick’  reduces  most  uncertainty,  and  it  would  therefore 
be  optimal  to  request  this  symptom  first. 

Information  integration  is  indicated  by  the  difference  between  the  probability  of 
the  chosen  option  given  the  requested  information  and  the  probability  of  the 
best  option  given  the  requested  information.  Thus,  when  the  best  option  given 
the  requested  information  is  selected,  this  value  is  0. 

Processing  time:  time  span  between  last  information  request  and  the  selection  of 
a  treatment.  In  this  time  span  subjects  have  to  combine  the  symptom 
information,  deduce  the  most  probable  cause  of  the  decline  and  implement  an 
action. 

Performance  is  indicated  by  the  overall  money  level  af  the  end  of  the 
experiment,  reflecting  the  trade-off  between  the  costs  anu  .he  risk  of  athlete 
collapse  and  by  the  number  of  correct  athlete  recoveries. 


3  RESULTS 

3.1  Learning  the  relations  between  symptoms  and  causes 

All  subjects  were  able  to  learn  the  relations  between  symptoms  and  causes 
(mean  number  of  trials  was  205  (s=53.7),  and  the  mean  score  in  the  last  run  was 
97.3  (s=6.4)). 

The  development  of  the  learning  process  is  shown  in  Figure  4. 
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Fig.  4  Learning  curve  expressing  the  percentage  correct  trials  over 
successive  runs.  A  run  comprises  ten  judgements  of  symptom 
combinations. 


3.2  Experimental  data 
Timing 

The  relative  times  (time  until  interference  divided  by  the  total  amount  of  time 
available)  were  affected  by  the  amount  of  time  pressure  that  was  imposed  on  the 
subjects  (F(l,36)  =  7.70,  p<.0l,  see  Figure  5):  under  high  levels  of  time  pressure 
subjects  started  to  interfere  sooner  with  the  athlete.  The  amount  of  training  did 
not  affect  the  moment  of  interference  (F(1,36)<1). 
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time  pressure 


Fig.  5  Relative  time  that  subjects  waited  after  the  onset  of  a  fitness 
decline  before  interference  by  either  an  information  request  or  an 
action  as  a  function  of  time  pressure  (low  or  high)  and  training  (low 
or  high).  The  relative  time  is  the  time  subjects  waited  divided  by  the 
total  amount  of  time  available. 


Decision  strategy 

Action-  and  judgment-oriented  strategies  were  used  equally  often  by  the  subjects 
and  were  not  affected  by  time  pressure  (F(1,36)<1,  proportion  action-oriented 
strategies  was  .50  in  the  low  time  pressure  condition  and  .50  in  the  high  time 
pressure  condition)  and  by  the  amount  of  training  (F(1,36)<1,  proportion  action 
oriented-strategies  was  .47  in  the  low  training  condition  and  .53  in  the  high 
training  condition).  Seven  (of  the  20)  subjects  in  each  training  condition  did  not 
request  information  at  all,  and  therefore  used  a  straight  action-oriented  strategy, 
and  seven  subjects  in  the  low  training  condition  and  nine  subjects  in  the  high 
training  condition  always  started  to  request  information  after  the  onset  of  a 
fltness  decline,  i.e.  used  a  judgment-oriented  strategy.  The  rest  of  the  subjects 
used  a  combination  of  both  strategies  over  the  different  trials. 

Information  search 

Subjects  requested  an  equal  amount  of  information  in  all  experimental 
conditions  (time  pressure:  F(1,36)<1,  training:  F(  1,36)  =  1.26,  p>.2).  Table  I 
shows  the  proportion  of  symptoms  that  were  first  requested  after  the  onset  of  a 
decline,  which  provides  an  indication  for  the  accuracy  of  the  selected 
information. 
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Table  I  Symptoms  that  were  requested  first  after  the  onset  of  a 
fitness  decline  as  a  function  of  time  pressure  (low  or  high)  and 
training  (low  or  high). 


Hlflll 

time  pressure  low 

time  pressure  high 

HH 

training  low 

training  high 

training  low 

training  high 

red 

.32 

.06 

.27 

.14 

sick 

.34 

.62 

.44 

.66 

tired 

.12 

0 

.16 

.03 

thirsty 

.23 

.32 

.14 

.16 

The  various  symptoms  were  not  equally  often  requested  (F(3,66)  =  7.4,  p<,0001), 
and  from  the  table  it  can  be  inferred  that  subjects  mostly  requested  the  symptom 
’sick’.  A  marginally  significant  interaction  was  found  between  the  kind  of 
information  requested  and  the  training  condition  (F(3,66)=2.5,  p  =  .07);  subjects 
who  had  a  more  elaborate  training  requested  the  symptom  ’sick’  more  often, 
implying  a  more  optimal  information  selection  process.  Overall,  this  result 
indicates  that  subjects  were  well  aware  of  the  value  of  the  various  symptoms, 
since  they  mostly  selected  the  symptom  that  would  reduce  most  uncertainty 
(sick). 

Information  integration 

No  main  effects  of  training  (F(1,36)<1)  and  time  pressure  (F(l,36)=2,69,  p>.l) 
were  found  on  the  accuracy  of  the  integration  of  information,  only  a  marginal 
interaction  between  time  pressure  and  training  (F(  1,36) =3.63,  p  =  .07,  see  Figure 
6).  Subjects  who  had  only  limited  training  performed  worse  when  time  pressure 
increased. 
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Fig.  6  Distance  between  chosen  hypothesis  and  optimal  hypothesis 
(as  indicated  by  the  a  posteriori  probability)  as  a  function  of  time 
pressure  (low  or  high)  and  training  (low  or  high).  A  value  of  0  means, 
that  the  optimal  hypothesis,  given  the  requested  information,  was 
chosen. 


Processing  time 

Subjects  who  experienced  more  time  pressure  were  faster  than  the  subjects  who 
experienced  less  time  pressure  (F(  1,22)  =  10.96,  p<.005),  and  subjects  who  had 
elaborate  training  were  faster  than  the  subjects  with  only  limited  training 
(F( 1,22) =4.45,  p<.05,  see  Figure  7).  Note  that  only  subjects  who  had  requested 
information  could  be  used  for  this  analysis,  meaning  that  7  subjects  in  each 
training  condition  were  excluded. 
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Fig.  7  Time  span  (in  seconds)  between  the  last  information  request 
and  the  succeeding  action  as  a  function  of  time  pressure  (low  or  high) 
and  training  (low  or  high). 


Performance 

Subjects  who  were  more  extensively  trained  earned  more  money  than  the 
subjects  with  less  training  (F(  1,36) =4.56,  p<.05,  97  guilders  versus  84  guilders). 
Thus,  with  more  training  subjects  made  better  trade-offs  between  the  various 
costs  and  risks  on  athlete  collapse.  It  did  not  matter  whether  subjects 
experienced  high  or  low  time  pressure  (F(  1,36) =2.82,  p>.l). 

Under  a  low  level  of  time  pressure  more  athletes  were  recovered  than  under  a 
high  level  of  time  pressure  (F(  1,36) =6.46,  p<.05,  see  Figure  8),  and  the  well- 
trained  subjects  made  more  correct  athlete  recoveries  than  the  subjects  with  only 
minimal  training  (F(  1,36) =5.8,  p<.05). 
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Fig.  8  Proportion  correct  recoveries,  meaning  that  fitness  declines 
due  to  physiological  causes  were  correctly  restored,  as  a  function  of 
time  pre.ssure  (low  or  high)  and  training  (low  or  high). 


4  DISCUSSION 

In  contrast  with  my  prediction,  training  did  not  affect  the  decision  strategy  that 
was  dominantly  used:  in  both  training  conditions  about  50%  of  the  subjects  used 
an  action-oriented  strategy,  and  the  other  half  used  a  judgment-oriented  strategy. 
As  the  use  of  an  action-oriented  strate^  would  lead  to  optimal  performance, 
with  minimal  effort,  this  result  implies  that  subjects  did  not  adaptively  react  to 
the  task  dynamics.  Half  of  the  subjects  in  the  present  experiment  received  quite 
exten.  ve  training,  and  continuous  feedback,  but  they  still  used  the  same 
strate;  es  as  the  subjects  who  did  not  have  such  an  elaborate  training. 
Insufficient  experience  with  the  task  seems  therefore  no  plausible  explanation  for 
the  use  of  less  than  optimal  strategies. 

Rules  for  the  selection  of  an  appropriate  strategy  for  performing  a  task  are 
referred  to  as  ’metastrategies’,  specifying  how  one  decides  to  tackle  the  problem 
at  hand.  One  possibility,  which  could  be  put  forward  as  an  explanation  for  the 
rather  consistent  use  of  a  judgment-oriented  strategy,  is  that  the  global  task 
features  trigger  a  general  class  of  strategies.  Diagnosis  tasks,  which  I  also  used  in 
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the  present  experiment,  could  then  well  be  associated  with  a  general  class  off 
’infer-than-act’  strategies  (KJeinmuntz  and  Thomas,  1987).  Thus,  it  is  suggested 
that  the  gencial  strategy  is  based  upon  the  global  task  characteristics  and  that 
adaptivity  occurs  within  the  constraints  of  this  strategy,  such  as  speeding  up 
processing  or  limiting  the  number  of  information  requests. 

Still,  the  number  of  subjects  using  an  action-oriented  strategy  was  considerably 
higher  than  in  our  previous  experiments,  in  which  judgment-oriented  strategies 
were  used  in  about  75%  of  the  trials  (Kerstholt,  submitted).  Two  reasons  may 
explain  this  difference.  First,  the  training  program  for  learning  the  relations 
between  symptoms  and  causes  was  changed,  such  that  the  subjects  in  the  present 
experiment  received  both  feedback  on  the  accuracy  of  their  chosen  option  with 
respect  to  the  requested  information,  i.e.  the  a  posteriori  probability,  and  on  the 
accuracy  of  the  chosen  option  with  regard  to  the  actual  reason  of  the  decline.  In 
the  previous  experiments  the  subjects  only  received  feedback  on  the  accuracy 
with  regard  to  the  a  posteriori  probability.  Providing  both  forms  of  feedback 
allowed  the  subjects  to  experience  the  limitations  of  the  value  of  the 
information;  even  when  they  knew  all  symptoms  and  deduced  the  most  optimal 
hypothesis,  the  athlete  could  still  suffer  from  a  different  disorder. 

Second,  in  the  previous  experiments  subjects  only  received  a  minimum  number 
of  trials  to  familiarize  themselves  with  the  experimental  task.  As  a  matter  of  fact, 
it  was  left  to  the  judgment  of  the  experimenter  to  determine  the  moment  the 
actual  experiment  would  start,  depending  on  the  abilities  of  the  subjects.  In  the 
present  experiment,  however,  subjects  trained  at  least  15  minutes.  Even  though 
these  subjects  did  not  receive  feedback  on  the  money  that  had  been  earned,  they 
did  receive  feedback  on  the  accuracy  of  their  actions  by  observing  the  athlete’s 
fitness  level.  This  information  may  have  been  sufficient  for  subjects  to  partly 
switch  to  an  ’action-oriented’  strategy. 

Even  though  strategy  selection  appears  to  be  independent  from  the  amount  of 
training,  subjects  with  more  elaborate  training  had  a  better  outcome 
performance,  such  that  they  ended  with  a  larger  amount  of  money  at  the  end  of 
the  experimental  session.  As  noted  above  this  difference  cannot  be  ascribed  to 
the  selection  of  a  different  strategy.  Rather,  the  differences  appear  at  a  lower 
behavioral  level;  more  training  resulted  in  more  accurate  information  search  and 
faster  processing.  The  amount  of  training  did  not  affect  the  accuracy  of 
information  integration.  In  a  recent  overview  of  information  use  by  experts, 
Shanteau  (1992)  concluded  that  research  consistently  indicated  that  experts  used 
as  much  information  as  novices.  He  suggested,  therefore,  that  it  is  the  kind  of 
information  that  distinguishes  experts  from  novices,  rather  than  the  amount  of 
information  (see  also  Johnson,  Duran,  Hassebrock,  Moller,  Prietula,  Feltovich 
and  Swanson,  1981,  for  a  similar  conclusion).  This  suggestion  is  supported  by  the 
present  findings;  even  though  the  subjects  in  both  training  conditions  requested 
the  same  amount  of  information,  the  well-trained  subjects  selected  more 
diagnostic  information,  as  indicated  by  the  amount  of  uncertainty  reduction. 
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The  time  pressure  effects  replicate  previous  findings;  when  there  was  less  time 
for  diagnosis  subjects  reacted  sooner  to  a  fitness  decline  (Kerstholt,  submitted), 
they  used  the  same  decision  strategy  (Kerstholt,  in  press,  submitted)  and  they 
speeded  up  information  processing  (Edland  and  Svenson,  1993;  Kerstholt, 
submitted;  Payne,  Bettman  and  Johnson,  1988).  An  explanation  for  the  absence 
of  strategic  effects  in  previous  experiments  using  an  within-subjects  design  could 
have  been  that  subjects  had  learned  a  strategy  that  would  marginally  fit  all  time 
conditions.  Note,  however,  that  we  used  a  between-subjects  design  in  the  present 
experiment,  implying  that  subjects  could  select  the  strategy  that  would  optimally 
fit  one  single  time  condition.  As  the  same  strategy  was  used  in  each  time 
condition  when  using  a  between-subjects  design,  we  may  conclude  that  a  strategy 
switch  is  not  in  the  behavioral  repertoire  to  respond  to  time  pressure,  and  that 
the  absence  of  time  pressure  effects  on  strategy  selection  is  not  due  to  the  use  of 
a  within-subjects  design. 

Subjects  who  had  only  limited  training  performed  less  well  when  time  pressure 
increased  than  subjects  who  had  had  an  extensive  training:  tney  applied  more 
incorrect  treatments.  This  performance  deterioration  for  the  less  well  trained 
group  was  due  to  less  accurate  information  integration  processes.  Thus, 
information  search  remains  constant  for  both  training  groups  over  time  pressure 
conditions,  but  the  integration  processes  deteriorated  for  the  subjects  who  had 
only  limited  training.  The  computational  requirements  for  the  integration  of 
probabilistic  information  into  one  overall  judgment  of  the  situation  provides  a 
heavy  load  on  working  memory,  which  is  generally  considered  as  a  system  with 
limited  capacity.  Thus,  the  present  results  suggest  that  in  addition  to  the 
deteriorating  effects  of  stressors  such  as  anxiety  (Hockey,  1986),  time  pressure 
affects  task  components  with  a  high  working  memory  demand  as  well.  These 
deteriorating  effects  can  be  overcome,  however,  with  training.  Plausibly,  training 
reduces  the  working  memory  load  as  processing  is  shifted  towards 
preprogrammed  procedures,  or  in  other  words  decision  making  may  become 
more  recognition  primed  (Klein,  1992). 

To  conclude,  the  results  of  the  experiment  showed  that  training  does  not  affect 
the  overall  strate©'  that  subjects  used  for  the  decision  task.  However,  subjects 
with  more  extensive  training  performed  better,  which  was  due  to  more  accurate 
information  search  processes  and  to  faster  processing.  In  line  with  previous 
findings  subjects  did  not  change  their  decision  strategy  as  a  consequence  of  time 
pressure,  even  though  a  between-subjects  design  was  used  in  the  present 
experiment.  Furthermore,  elaborate  training  can  make  the  integration  of 
information  more  resistant  to  negative  effects  of  time  pressure. 
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